Identifying and localizing a vehicle occupant by correlating hand gesture and seatbelt motion

ABSTRACT

A method, system, and vehicle for locating an occupant within the vehicle. The method includes receiving motion data of a seatbelt for a seat in the vehicle and receiving motion data of a wearable device of the occupant. The method also includes comparing the seatbelt motion data with the wearable device motion data from a same period of time and identifying that the occupant is located at the seat in the vehicle based on a result of the comparison. The method may also include associating the occupant with the vehicle based on location information of the vehicle and location information of the wearable device of the occupant. Additionally, the method may include identifying personalization options for the occupant at the seat based on a user profile of the occupant in response to identifying that the occupant is located at the seat.

TECHNICAL FIELD

This disclosure relates generally to vehicle personalization systems.More specifically, this disclosure relates to identifying and localizinga vehicle occupant by correlating hand gesture and seatbelt motion.

BACKGROUND

The connected car, with increasing annual revenue, has attracted strongresearch and development interest from automakers, mobile carriers, andconsumer electronics manufactures. Among the research and developmentfocuses, a key challenge is to enable personalized services such asinformation and entertainment (infotainment) for each occupant,including both the driver and passengers. Wearable devices, such asSamsung Gear series smartwatches, have become increasingly popular.These wearable devices can provide activity tracking and other functionscomparable to those provided by a smart phone.

Mobile carriers have recently launched applications and services torestrict phone use while driving. Specifically, if a phone is detectedto be inside a moving car, it will limit functionalities to avoidpossible distractions, regardless whether it is with the driver orpassengers. This is inconvenient for passengers, who will usually end updisabling this service all together.

SUMMARY

Embodiments of the present disclosure provide for identifying andlocalizing a vehicle occupant by correlating hand gesture and seatbeltmotion.

In one embodiment, a system for locating an occupant within a vehicle isprovided. The system includes a communication interface and at least oneprocessor. The communication interface is configured to receive motiondata of a seatbelt for a seat in the vehicle and receive motion data ofa wearable device of the occupant. The at least one processor isconfigured to compare the seatbelt motion data with the wearable devicemotion data from a same period of time and identify that the occupant islocated at the seat in the vehicle based on a result of the comparison.

In another embodiment, a vehicle for locating an occupant is provided.The vehicle includes one or more seats; one or more seatbelts for theone or more seats; a communication interface, and at least oneprocessor. The communication interface is configured to receive motiondata of one of the seatbelts, send the seatbelt motion data to a serverfor comparison with motion data of a wearable device of the occupant ofthe vehicle from a same period of time, and receive informationindicating that the occupant is located at a seat of the one seatbelt.The at least one processor is configured to identify personalizationoptions for the occupant at the seat based on a user profile of theoccupant in response to identification that the occupant is located atthe seat.

In yet another embodiment, a method for locating an occupant within avehicle is provided. The method includes receiving motion data of aseatbelt for a seat in the vehicle and receiving motion data of awearable device of the occupant. The method also includes comparing theseatbelt motion data with the wearable device motion data from a sameperiod of time and identifying that the occupant is located at the seatin the vehicle based on a result of the comparison.

Other technical features may be readily apparent to one skilled in theart from the following figures, descriptions, and claims.

Before undertaking the DETAILED DESCRIPTION below, it may beadvantageous to set forth definitions of certain words and phrases usedthroughout this patent document. The term “couple” and its derivativesrefer to any direct or indirect communication between two or moreelements, whether or not those elements are in physical contact with oneanother. The terms “transmit,” “receive,” and “communicate,” as well asderivatives thereof, encompass both direct and indirect communication.The terms “include” and “comprise,” as well as derivatives thereof, meaninclusion without limitation. The term “or” is inclusive, meaningand/or. The phrase “associated with,” as well as derivatives thereof,means to include, be included within, interconnect with, contain, becontained within, connect to or with, couple to or with, be communicablewith, cooperate with, interleave, juxtapose, be proximate to, be boundto or with, have, have a property of, have a relationship to or with, orthe like. The term “controller” means any device, system or part thereofthat controls at least one operation. Such a controller may beimplemented in hardware or a combination of hardware and software and/orfirmware. The functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely. Thephrase “at least one of,” when used with a list of items, means thatdifferent combinations of one or more of the listed items may be used,and only one item in the list may be needed. For example, “at least oneof: A, B, and C” includes any of the following combinations: A, B, C, Aand B, A and C, B and C, and A and B and C.

Moreover, various functions described below can be implemented orsupported by one or more computer programs, each of which is formed fromcomputer readable program code and embodied in a computer readablemedium. The terms “application” and “program” refer to one or morecomputer programs, software components, sets of instructions,procedures, functions, objects, classes, instances, related data, or aportion thereof adapted for implementation in a suitable computerreadable program code. The phrase “computer readable program code”includes any type of computer code, including source code, object code,and executable code. The phrase “computer readable medium” includes anytype of medium capable of being accessed by a computer, such as readonly memory (ROM), random access memory (RAM), a hard disk drive, acompact disc (CD), a digital video disc (DVD), or any other type ofmemory. A “non-transitory” computer readable medium excludes wired,wireless, optical, or other communication links that transporttransitory electrical or other signals. A non-transitory computerreadable medium includes media where data can be permanently stored andmedia where data can be stored and later overwritten, such as arewritable optical disc or an erasable memory device.

Definitions for other certain words and phrases are provided throughoutthis patent document. Those of ordinary skill in the art shouldunderstand that in many if not most instances, such definitions apply toprior as well as future uses of such defined words and phrases.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure and itsadvantages, reference is now made to the following description taken inconjunction with the accompanying drawings, in which like referencenumerals represent like parts:

FIG. 1 illustrates an example communication system in which variousembodiments of the present disclosure may be implemented;

FIG. 2 illustrates an example computer system according to variousembodiments of the present disclosure;

FIG. 3 illustrates an example electronic device according to variousembodiments of the present disclosure;

FIG. 4 illustrates a block diagram of an example vehicle for providingpersonalization services for located occupants according to oneembodiment of this disclosure;

FIG. 5 illustrates a block diagram of an example enhanced seatbeltbuckle according to various embodiments of the present disclosure;

FIG. 6 illustrates an example enhanced seatbelt buckle according to oneembodiment of the present disclosure;

FIG. 7 illustrates an example sensor fusion pipeline process forobtaining a device rotation matrix according to one embodiment of thepresent disclosure;

FIG. 8 illustrates an example process for extract ground-based motionfeatures according to one embodiment of the present disclosure;

FIG. 9 illustrates a graph including a comparison of exampleground-based acceleration features;

FIG. 10 illustrates a flowchart of an example process for occupantidentification and personalization according to one embodiment of thepresent disclosure; and

FIG. 11 illustrates a flowchart for a process for locating an occupantwithin a vehicle according to various embodiments of this disclosure.

DETAILED DESCRIPTION

FIGS. 1 through 10, discussed below, and the various embodiments used todescribe the principles of the present disclosure in this patentdocument are by way of illustration only and should not be construed inany way to limit the scope of the disclosure. Those skilled in the artwill understand that the principles of the present disclosure may beimplemented in any suitably arranged system or device.

Embodiments of the present disclosure recognize that a major challengein connected car solutions is how to enable personalizationautomatically. For example, the driver can have her favorite musicchannel and seat position set up without her taking additional actions.Moreover, if the vehicle has an entertainment system for backseatpassages (e.g., display, audio), further personalization is useful. Toenable complete personalization for each seat, the vehicle needs to knowthe identity of the occupant in each seat. Accordingly, embodiments ofthe present disclosure provide for identification of the occupant ineach seat automatically or without the occupant needing to enteridentifying information herself. Various embodiments of the presentdisclosure provide support for occupant personalization without requireoccupant's intervention for identification. Some detailed examples areprovided below.

Embodiments of the present disclosure recognize that progress in driveridentification has occurred. Some cars identify the driver based on thekey inserted. However, embodiments of the present disclosure recognizethat this method is not applicable if the key is shared by multipledrivers, (e.g., in the case of rental cars). Some other cars identifythe driver by asking for an identifier (e.g., a PIN). Embodiments of thepresent disclosure recognize that this method is inconvenient as itrelies on driver's interaction.

Embodiments of the present disclosure also recognize that other optionsfor identifying an occupant in each seat of a vehicle involve requestoccupants to take addition actions. In particular, an occupant canidentify herself through barcode labels, RFID tags, or biometriccollection devices (e.g., camera, fingerprint reader, etc.) at her seat.Embodiments of the present disclosure recognize that these options maybe undesirable requiring intervention from each occupant. Moreover,privacy issues become prevalent when biometrics are used foridentification.

Accordingly, embodiments of the present disclosure provide convenienceby using wearable devices provide personalization in connected car.Embodiments of the present disclosure address the aforementioned issuesby making the vehicle aware of the identity of occupant in each seatwithout requiring the occupant's intervention.

FIG. 1 illustrates an example communication system 100 in which variousembodiments of the present disclosure may be implemented. The embodimentof the communication system 100 shown in FIG. 1 is for illustrationonly. Other embodiments of the communication system 100 could be usedwithout departing from the scope of this disclosure.

As shown in FIG. 1, the system 100 includes a network 102, whichfacilitates communication between various components in the system 100.For example, the network 102 may communicate Internet Protocol (IP)packets, frame relay frames, or other information between networkaddresses. The network 102 may include one or more local area networks(LANs); metropolitan area networks (MANs); wide area networks (WANs);all or a portion of a global network, such as the Internet; or any othercommunication system or systems at one or more locations.

The network 102 facilitates communications between at least one server104 and various client devices 106-110. Each server 104 includes anysuitable computing or processing device that can provide computingservices for one or more client devices. Each server 104 could, forexample, include one or more processing devices, one or more memoriesstoring instructions and data, and one or more network interfacesfacilitating communication over the network 102.

Each client device 106-110 represents any suitable computing orcommunication device with a display that interacts with at least oneserver or other computing device(s) over the network 102. In thisexample, the client devices 106-114 include electronic devices, such as,for example, mobile telephone(s) or smartphone(s) 106, wearabledevice(s) 108, vehicle(s) 110, a laptop computer, a tablet computer,etc. However, any other or additional client devices could be used inthe communication system 100.

In this example, some client devices 106-110 communicate indirectly withthe network 102. For example, the client devices 106-110 communicate viaone or more base stations 116, such as cellular base stations oreNodeBs. Also, the client devices 106-110 may communicate via one ormore wireless access points (APs), such as IEEE 802.11 wireless APs.Note that these are for illustration only and that each client devicecould communicate directly with the network 102 or indirectly with thenetwork 102 via any suitable intermediate device(s) or network(s). Forexample, the vehicle 110 may communicate with the network 102 via one ormore satellites 118 for receiving and/or sending entertainment andlocation (e.g., GPS) information.

In this illustrative example, vehicle 110 is a connected vehicle capableof providing information and entertainment to occupants. As described inmore detail below, the vehicle 110 provides personalization options forthe occupants based on a user profile as a result of locating theoccupants within seats of the vehicle 110. The vehicle 110 alsocommunicates with devices (e.g., wearable devices 108 and enhancedseatbelt buckles) located within the vehicle 110. For example, thevehicle 110 may communicate via a personal area network (PAN), such asBluetooth, or near field communication (NFC) to send and receiveinformation to and from devices located within the vicinity of vehicle110. While illustrated as a car, vehicle 110 may be any suitable vehiclecapable of communication and for which personalization options can beprovided for occupants, such as, for example, without limitation, a bus,a train, a plane, etc.

As described in more detail below, the server 104 may represent asserver of a cloud computing system that provides for the identificationand localization of a vehicle occupant by correlating hand gesture andseatbelt motion. For example, the server 104 may receive motion patternsfrom a wearable device 108 of an occupant of the vehicle 110 as well asfrom an enhanced seatbelt buckle 112 (e.g., the connected vehicle 110).

Although FIG. 1 illustrates one example of a communication system 100,various changes may be made to FIG. 1. For example, the system 100 couldinclude any number of each component in any suitable arrangement. Ingeneral, computing and communication systems come in a wide variety ofconfigurations, and FIG. 1 does not limit the scope of this disclosureto any particular configuration. While FIG. 1 illustrates oneoperational environment in which various features disclosed in thispatent document can be used, these features could be used in any othersuitable system.

FIGS. 2 and 3 illustrate example electronic devices in a communicationsystem according to various embodiments of the present disclosure. Inparticular, FIG. 2 illustrates an example computer system 200, and FIG.3 illustrates an example electronic device 300. In this illustrativeexample, the computer system 200 represents the server 104 in FIG. 1,and the electronic device 300 could represent one or more of the clientdevices 106-108 in FIG. 1.

As shown in FIG. 2, the computer system 200 includes a bus system 205,which supports communication between at least one processor 210, atleast one storage device 215, at least one communications interface 220,and at least one input/output (I/O) unit 225.

The processor 210 executes instructions that may be loaded into a memory230. The processor 210 may include any suitable number(s) and type(s) ofprocessors or other devices in any suitable arrangement. Example typesof processor 210 include microprocessors, microcontrollers, digitalsignal processors, field programmable gate arrays, application specificintegrated circuits, and discreet circuitry. The processor 210 may be ageneral-purpose CPU or specific purpose processor for encoding ordecoding of video data.

The memory 230 and a persistent storage 235 are examples of storagedevices 215, which represent any structure(s) capable of storing andfacilitating retrieval of information (such as data, program code,and/or other suitable information on a temporary or permanent basis).The memory 230 may represent a random access memory or any othersuitable volatile or non-volatile storage device(s). The persistentstorage 235 may contain one or more components or devices supportinglonger-term storage of data, such as a read-only memory, hard drive,Flash memory, or optical disc.

The communications interface 220 supports communications with othersystems or devices. For example, the communications interface 220 couldinclude a network interface card or a wireless transceiver (e.g.,satellite, cellular, WiFi, Bluetooth, NFC, etc.) facilitatingcommunications over the network 102. The communications interface 220may support communications through any suitable physical or wirelesscommunication link(s). The communications interface 220 may include onlyone or both of a transmitter and receiver, for example, only a receivermay be included in a decoder or only a transmitter may be included in anencoder.

The I/O unit 225 allows for input and output of data. For example, theI/O unit 225 may provide a connection for user input through a keyboard,mouse, keypad, touchscreen, or other suitable input device. The I/O unit225 may also send output to a display, printer, or other suitable outputdevice.

As described in more detail below, the computer system 200 may be aserver of a cloud computing system that provides for the identificationand localization of a vehicle occupant by correlating hand gesture andseatbelt motion. The computer system 200 may also be located in avehicle (e.g., such as vehicle 110) for locating an occupant and/orproviding personalization options for occupants located in the vehicle.

FIG. 3 illustrates an example electronic device 300 according to variousembodiments of the present disclosure. In this embodiment, theelectronic device 300 is an example of one or more of the client devices(e.g., such as client devices 106-108 of FIG. 1). The embodiment of theelectronic device 300 illustrated in FIG. 3 is for illustration only,and the client devices 106-108 of FIG. 1 could have the same or similarconfiguration. However, electronic devices come in a wide variety ofconfigurations, and FIG. 3 does not limit the scope of this disclosureto any particular implementation of an electronic device.

As shown in FIG. 3, the electronic device 300 includes antenna(s) 305, atransceiver 310, transmit (TX) processing circuitry 315, a microphone320, and receive (RX) processing circuitry 325. The electronic device300 also includes a speaker 330, a processor 340, an input/output (I/O)interface (IF) 345, a touchscreen 350, a display 355, a memory 360, andone or more sensors 365. The memory 360 includes an operating system(OS) 361 and one or more applications 362.

The transceiver 310 receives, from the antenna(s) 305, an incoming RFsignal transmitted by an access point (e.g., base station, WiFi router,Bluetooth device) for a network (e.g., a WiFi, Bluetooth, cellular, 5G,LTE, LTE-A, WiMAX, or any other type of wireless network). Thetransceiver 310 down-converts the incoming RF signal to generate anintermediate frequency (IF) or baseband signal. The IF or basebandsignal is sent to the RX processing circuitry 325, which generates aprocessed baseband signal by filtering, decoding, and/or digitizing thebaseband or IF signal. The RX processing circuitry 325 transmits theprocessed baseband signal to the speaker 330 (such as for voice data) orto the processor 340 for further processing (such as for web browsingdata).

The TX processing circuitry 315 receives analog or digital voice datafrom the microphone 320 or other outgoing baseband data (such as webdata, e-mail, or interactive video game data) from the processor 340.The TX processing circuitry 315 encodes, multiplexes, and/or digitizesthe outgoing baseband data to generate a processed baseband or IFsignal. The transceiver 310 receives the outgoing processed baseband orIF signal from the TX processing circuitry 315 and up-converts thebaseband or IF signal to an RF signal that is transmitted via theantenna(s) 305. In some embodiments, the transceiver 310, the TXprocessing circuitry 315, and the RX processing circuitry 325 may beimplemented within the same block or chip, such as a WiFi and/orBluetooth module and/or chip.

The processor 340 can include one or more processors or other processingdevices and execute the OS 361 stored in the memory 360 in order tocontrol the overall operation of the electronic device 300. For example,the processor 340 could control the reception of forward channel signalsand the transmission of reverse channel signals by the transceiver 310,the RX processing circuitry 325, and the TX processing circuitry 315 inaccordance with well-known principles. In some embodiments, theprocessor 340 includes at least one microprocessor or microcontroller.

The processor 340 is also capable of executing other processes andprograms resident in the memory 360. The processor 340 can move datainto or out of the memory 360 as required by an executing process. Insome embodiments, the processor 340 is configured to execute theapplications 362 based on the OS 361 or in response to signals receivedfrom eNBs or an operator. The processor 340 is also coupled to the I/Ointerface 345, which provides the electronic device 300 with the abilityto connect to other devices, such as laptop computers and handheldcomputers. The I/O interface 345 is the communication path between theseaccessories and the processor 340.

The processor 340 is also coupled to the touchscreen 350 and the display355. The operator of the electronic device 300 can use the touchscreen350 to enter data and/inputs into the electronic device 300. The display355 may be a liquid crystal display, light-emitting diode (LED) display,optical LED (OLED), active matrix OLED (AMOLED), or other displaycapable of rendering text and/or at graphics, such as from web sites,videos, games, etc. The touchscreen 350 can include a touch panel, a(digital) pen sensor, a key, or an ultrasonic input device.

The memory 360 is coupled to the processor 340. Part of the memory 360could include a random access memory (RAM), and another part of thememory 360 could include a Flash memory or other read-only memory (ROM).

Electronic device 300 further includes one or more sensors 365 that canmeter a physical quantity or detect an activation state of theelectronic device 300, and convert metered or detected information intoan electrical signal. For example, sensor 365 may include one or morebuttons for touch input, a camera, a gesture sensor, a gyroscope or gyrosensor, an air pressure sensor, a magnetic field sensor or magnetometer,an acceleration sensor or accelerometer, a grip sensor, a proximitysensor, a color sensor (e.g., a Red Green Blue (RGB) sensor), atemperature/humidity sensor, an illumination sensor, etc. The sensor(s)365 can further include a control circuit for controlling at least oneof the sensors included therein. As will be discussed in greater detailbelow, one or more of these sensor(s) 365 may be one or more of anaccelerometer, a gyroscope, and a magnetic field sensor to generate amotion pattern for an occupant buckling her seatbelt.

Although FIG. 3 illustrates one example of an electronic device 300,various changes may be made to FIG. 3. For example, various componentsin FIG. 3 could be combined, further subdivided, or omitted andadditional components could be added according to particular needs. As aparticular example, the processor 340 could be divided into multipleprocessors, such as one or more central processing units (CPUs) and oneor more graphics processing units (GPUs). In one or more embodiments,the speaker 330, the microphone 320, and or the touchscreen 350 may notbe included in the device 300. For example, in some embodiments, theelectronic device 300 may be a smartwatch having display/user interfacefunctionality comparable with that of a smartphone or may be a simpleactivity tracker with wireless communication functionality but limitedor no display/user interface functionality.

Embodiments of the present disclosure recognize and take into accountthat each occupant of a vehicle 110 makes a unique gesture to buckle herseatbelt, which includes raising her hand to grab the seatbelt, pullingthe seatbelt around her waist, inserting the buckle into the latch, andfinally moving the hand away after the click. This entire gesture can becaptured by occupant's wearable device 108 (e.g., a smartwatch oractivity tracker) on her hand. Based on the assumption that the seatbeltsensors in the enhanced seatbelt buckle 112 (as discussed below) cansimultaneously capture the corresponding movement during this gesture,embodiments of the present disclosure correlate these two movementpatterns to uniquely identify the occupant of each seat, thus supportingpersonalization for all occupants.

Components of various embodiments of the present disclosure include anenhanced seatbelt buckle 112 to capture the seatbelt's movement duringthe seatbelt buckling gesture; a wearable device 108 worn by an occupantto capture her hand movement during the seatbelt buckling gesture; anoccupant identification method to associate the occupant's identitythrough her wearable device and the seat she occupies based oncorrelating the above two movement patterns; a connected vehicle foridentifying locations of occupants within the vehicle and providingpersonalization options based on occupant locations; and a cloud basedplatform to support personalization for each occupant.

FIG. 4 illustrates a block diagram of an example vehicle 110 forproviding personalization services for located occupants according toone embodiment of this disclosure. In this embodiment, the vehicle 110of FIG. 1 is illustrated with example components. The embodiment of thevehicle 110 illustrated in FIG. 4 is for illustration only. FIG. 4 doesnot limit the scope of this disclosure to any particular implementationof an electronic device.

As illustrated, the vehicle 110 includes a computer system 200, variousseats 405 for occupants (e.g., a driver and/or passengers), andpersonalization components 410 and 415. The personalization components410 and 415 are components of the vehicle that can provide personalizedservices to occupants of the vehicle. Personalized services includemedia content such as infotainment that is specific to the occupant. Forexample, the display 410 may display a home screen of applications orcontent from an occupant's mobile device or preset infotainment based onthe occupant's profile. The illustrated example includes displays 410and speakers 415; however, any type of personalization components may beused. For example, each seat 405 may have a separate display 410 andspeaker(s) 415 or the display 410 and speaker(s) 415 may be common formultiple seats 405. The display 410 may be a display in a dashboard, aconsole, or the back of another seat. In another example, the speaker415 may be a headphones or audio jack for providing audio to the seatoccupant.

The computer system 200 provides for communication and control ofcomponents within the vehicle 110. For example, the computer system 200may query and receive motion data from enhanced seatbelt buckles 112 andmay identify that the seats 405 are occupied based on an output from theseat sensor(s) 425. The computer system 200 can communicate thisinformation to a cloud connected server 104 and receive informationabout locations of occupants within the vehicle and occupant profiledata for personalization. The computer system 200 can also communicatewith user devices, such as, mobile phones 106 and wearable devices 108and control the output of infotainment via displays 410 and speakers415. In some embodiments, the computer system 200 may make a comparisonof motion data for the enhanced seatbelt buckles 425 and the userwearable devices 108 to determine the locations of occupants within thevehicle and occupant profile data for personalization.

FIG. 5 illustrates a block diagram of an example enhanced seat beltbuckle 112 according to various embodiments of the present disclosure.The embodiment of the enhanced seat belt buckle 112 shown in FIG. 5 isfor illustration only. Other embodiments of the enhanced seat beltbuckle 112 could be used without departing from the scope of thisdisclosure.

The enhanced seatbelt buckle 112 is an important component of variousembodiments of the present disclosure. The enhanced seatbelt buckle 112captures the seatbelt buckling gesture on the vehicle side. The enhancedseatbelt buckle 112 includes one or more motion sensor(s) 505 to capturethe movement of buckle 112 during the seatbelt buckling gesture. Themotion sensor(s) 505 may include, but are not limited to, motion sensorssuch as accelerometer and gyroscope. For example, the motion sensor(s)505 may include a gesture sensor, a gyroscope or gyro sensor, a magneticfield sensor or magnetometer, an acceleration sensor or accelerometer,etc. The memory 510 stores the captured motion pattern. Transceiver 515communicates motion pattern data to with the car. One or more of variouswireless communication protocols can be used, such as Wi-Fi, Bluetooth,ZigBee, and Z-Wave, for example. Controller 520 provides control tocoordinate and bridge different components of the buckle 112.Specifically, controller 520 processes the measurements of motionsensor(s) 505, stores measurements in the memory 510, and transmits themto the vehicle via the transceiver 515. The energy supply 525 suppliespower the above discussed components. For example, the energy supply 525may be a battery, a wire to energy supply of the vehicle 110 via theseatbelt or the latch, and/or may use induction charging, for example,when buckled or when in an unbuckled position.

In practice, the enhanced seatbelt buckle 112 may be either areplacement for existing seatbelt buckles (i.e., integrated within theseatbelt buckle) or an accessory that can be attached to existingseatbelt buckles. FIG. 6 illustrates an example enhanced seat beltbuckle 112 according to one embodiment of the present disclosure. Theembodiment of the enhanced seat belt buckle 112 shown in FIG. 6 is forillustration only. Other embodiments of the enhanced seat belt buckle112 could be used without departing from the scope of this disclosure.In this example, the enhanced seat belt buckle 112 implemented with TIMSP430 controller, 10 KB RAM memory, a CC2420 radio, a MPU-6050 sensor,and powered by two AA batteries.

Embodiments of the present disclosure extract robust motion features tocharacterize the seatbelt buckling gesture. One challenge is that motionsensors on mobile/wearable devices typically use the coordinate systemof the device. Thus, measurements of the device measurements aredependent on the device orientation and may not be able to be useddirectly. To address this issue, embodiments of the present disclosuremap the device coordinate system to the a coordinate system of theenvironment by computing a device rotation matrix, and then extractingmotion features based on the coordinate system of the environment.Alternatively, in practice, the motion features can be significantlysimplified as the scalar magnitudes of acceleration during a period ofmeasurement, thus the orientations of either seatbelt buckle or wearablebecome irrelevant.

FIG. 7 illustrates an example sensor fusion pipeline process 700 forobtaining a device rotation matrix according to one embodiment of thepresent disclosure. The process 700 may be performed by any componentthat processes the sensed motion data of the wearable device 108. Forexample, the server 104, the wearable device 108, and/or the computersystem 200, collectively referred to here as “the system.” Theembodiment of the sensor fusion pipeline process 700 shown in FIG. 7 isfor illustration only. Other embodiments of the sensor fusion pipelineprocess 700 could be used without departing from the scope of thisdisclosure.

In this illustrative example, the system computes device rotation matrixbased on sensor fusion. The system fuses the measurements ofaccelerometer 702, gyroscope 704, and magnetic field sensor 706 to getthe accurate device orientation 708. For example, the system performsnoise removal 710 to remove noise from the noisy orientation 712 of theaccelerometer 702 and magnetic field sensor 706 outputs. The system alsoaccounts for drifted orientation 714 of the gyroscope 704 using biasremoval 716. By comparing device orientation readings over time,embodiments of the present disclosure obtain device rotation matrix 718.In one example, the noise and bias removal 710 and 716 are implementedusing low-pass and high-pass filters, respectively.

FIG. 8 illustrates an example process 800 for extract ground-basedmotion features according to one embodiment of the present disclosure.The process 800 may be performed by any component that processes thesensed motion data of the wearable device 108. For example, the server104, the wearable device 108, and/or the computer system 200,collectively referred to here as “the system.” The embodiment of theprocess 800 shown in FIG. 8 is for illustration only. Other embodimentsof the process 800 could be used without departing from the scope ofthis disclosure.

FIG. 8 illustrates extraction of robust motion features. The systemconverts raw sensor measurements 802 from the device coordinate systemto the coordinate system of the environment using the rotation matrix718, thus extracting ground-based motion features 806. In addition, thesystem applies exponential smoothing to remove the bias and random noisein the raw sensor measurements 802.

FIG. 9 illustrates an example graph 900 including a comparison ofexample ground-based acceleration features. The example graph 900 shownin FIG. 9 is for illustration only. Other example graphs 900 could beused without departing from the scope of this disclosure.

In this illustrative example, the ground-based acceleration featuresextracted from three buckling gestures for the enhanced seatbelt buckle112 and wearable device 108 are compared during the same period of time.The buckling gesture maintains stable acceleration features along thethree directions in the coordinate system of the environment. Note thatsuch ground-based motion features do not depend on how the device iscarried.

Embodiments of the present disclosure identify an occupant bycorrelating the movement patterns captured by the enhanced seatbeltbuckle 112 and wearable device 108 during the same period of time. Insome embodiments, various metrics such as cosine similarity or otherthreshold correlation values may be adopted to quantify the correlation.Two movement patterns are considered as correlated if the movementpatterns captured by the enhanced seatbelt buckle 112 and wearabledevice 108 have matching timestamps and their similarity score is higherthan a certain threshold. FIG. 9 illustrates the acceleration featuressimultaneously captured the enhanced seatbelt buckle 112 and wearabledevice 108 during three buckling gestures. As illustrated, the twomovement patterns are well correlated.

FIG. 10 illustrates a flowchart of an example process 1000 for occupantidentification and personalization according to one embodiment of thepresent disclosure. In this embodiment, the parts of process depicted inFIG. 10 are performed by various components in the communication system100, including the cloud connected server 104; the wearable device 108,the vehicle 110, and the enhanced seatbelt buckle 112. The embodiment ofthe process 1000 shown in FIG. 10 is for illustration only. Otherembodiments of the process 1000 could be used without departing from thescope of this disclosure.

The process begins with the occupants (e.g., via the wearable device 108or the user's mobile phone) and the vehicle 110 providing their currentlocations (steps 1002 and 1004), respectively, to the cloud server 104.The server 104 associates a group of occupants to a vehicle based on theoccupants and vehicle location information (step 1006).

When occupant (e.g., the driver or a passenger) enters the vehicle 110and sits in a seat with the enhanced seatbelt buckle 112, the occupantwill buckle her seatbelt. During this gesture, the sensors of enhancedseatbelt buckle 112 capture the movement of buckle (step 1008) as afeature-time sequence, e.g., denoted by [(p₀ ^(CS), t₀), (p₁ ^(CS), t₁),. . . , (p_(N-1) ^(CS), t_(N-1))], in which feature p_(i) ^(CS) isground-based and may include the moving speed, acceleration, timing,and/or any other motion features of seatbelt buckle at time t_(i). Theseat belt buckling gesture is made by the occupant's hand wearing thewearable device 108. Simultaneously during this gesture, the motionsensors on wearable device 108 capture the movement of the occupant'shand (step 1010) as a feature-time sequence, e.g., denoted by [(P₀ ^(R),T₀), (P₁ ^(R), T₁), . . . , (P_(N-1) ^(R), T_(N-1))], in which featureP_(i) ^(R) is ground-based and may include the moving speed,acceleration, timing, and/or any other motion features of the occupant'shand at time T_(i).

In some embodiments, a set of machine learning algorithms, such assupport vector machine (SVM) and random forest (RF), are used todistinguish the seatbelt buckling gesture from other gestures.Therefore, in various embodiments, the features captured by the wearabledevice 108 may only characterize occupant's hand movement during thebuckling gesture.

When the seat is occupied, the vehicle 110 identifies that the seat isoccupied, for example, via an output from a seat sensor and sends anotification to the server 104 that the seat is occupied (step 1012).Upon detecting the seatbelt is buckled, the vehicle 110 queries for thecaptured movement (step 1014). The enhanced seatbelt buckle 112 sendsthe captured movement to the vehicle 110 (step 1016). Then, the vehicle110 uploads the motion data (e.g., the movement pattern) to the cloudserver 104 (step 1018).

Meanwhile, as a result of the vehicle 110 notifying the server 104 thatthe seat is occupied, the cloud then queries the wearable device 108 ofeach associated occupant for the captured motion data (step 1020). Eachwearable device 108 uploads the recently captured movement of bucklinggesture, if available, to the server 104 (step 1022).

The cloud server 104 compares the movement pattern from each wearabledevice 108 associated with the vehicle 110 to with that from the seatbuckle 112 to identify a pair of correlated movement patterns. Based onthe correlated movement patterns, the server determines that aparticular occupant R sits in a particular seat S of the vehicle 110.

The cloud server 104 notifies the vehicle 110 that occupant R is in seatS and then pushes (e.g., sends identification information) of occupantR's profile to the vehicle 110 (step 1026). As a result of being formedof the identity and location of the occupant R in the vehicle, thevehicle 110 provides personalized services at S based on the occupantR's profile.

These illustrative embodiments do not require the occupant'sintervention (e.g., input PIN and scan barcode) to identify occupant ineach seat. Rather, in these embodiments, the vehicle is enabled toautomatically load the occupant's personalized services before thevehicle starts to move.

FIG. 11 illustrates a process 1100 for locating an occupant within avehicle in accordance with various embodiments of the presentdisclosure. For example, the process depicted in FIG. 11 may beperformed by the server 104 in FIG. 1. The process may also beimplemented by the vehicle 110 in FIG. 1. The embodiment of the process1100 shown in FIG. 11 is for illustration only. Other embodiments of theprocess 1100 could be used without departing from the scope of thisdisclosure.

The process begins with the system receiving motion data of a seat beltfor a seat in the vehicle (step 1105). For example, in step 1105, thesystem may receive the seat belt motion data from the enhanced seatbuckle 112 via the vehicle 110. The system then receives motion data ofa wearable device of the occupant (step 1110). For example, in step1110, the system may receive, from the vehicle 110, a notification thatthe seat in the vehicle is occupied and then request or identifyreceived wearable device motion data for any occupant associated withthe vehicle in response to receipt of the notification. The system mayassociate the occupant with the vehicle 110 based on earlier receivedlocation information of the vehicle and location information of thewearable device of the occupant.

The system compares the seat belt motion data with the wearable devicemotion data (step 1115). For example, in step 1115, the motion data maybe processed to identify features of the motion from the respectivedevices for comparison. The system then identifies that the occupant islocated at the seat in the vehicle based on a result of the comparison(step 1120). For example, in step 1120, the system may identify a pairof correlated wearable device and seat belt motion patterns from among aplurality of wearable device motion patterns and seat belt motionpatterns for a plurality of occupants of the vehicle. The motion datamay be acceleration data and the system may then compare an accelerationpattern of the seat belt with an acceleration pattern of the wearabledevice.

Thereafter, the system identifies personalization options for theoccupant at the seat based on a user profile of the occupant (step1125). For example, in step 1125, the system may provide user profileinformation to the vehicle or the vehicle may load profile informationfrom a connected user device. Based on identifying that the occupant islocated at the seat and the user's profile information, the system mayload and provide personalized infotainment to the user in the vehicle(e.g., play audio or video, load applications, modify seat settings,etc.).

Although FIGS. 10 and 11 illustrate examples of processes for occupantidentification and personalization and locating an occupant within avehicle, respectively, various changes could be made to FIGS. 10 and 11.For example, while shown as a series of steps, various steps in eachfigure could overlap, occur in parallel, occur in a different order, oroccur multiple times.

Embodiments of the present disclosure provide an enhanced seatbeltbuckle to capture the movement of buckle during occupant's seatbeltbuckling gesture. Embodiments of the present disclosure characterize theseatbelt buckling gesture uses ground-based robust motion features. Bymonitoring and correlating the movement of occupant's hand and seatbeltbuckle, embodiments of the present disclosure can identify occupant ineach seat without occupant's intervention and thus enable herpersonalized services automatically. Embodiments of the presentdisclosure can be implemented in any other transportation systems thathave seatbelts and ability to support personalization.

Embodiments of the present disclosure address a key problem in connectedcar solutions—how to provide each occupant personalized servicesautomatically. By monitoring and correlating the movement of occupant'shand and seatbelt, embodiments of the present disclosure identify theoccupant in each seat without occupant's intervention and then enablespersonalization based on occupant's profile stored in the cloud or userdevice.

Although the present disclosure has been described with an exemplaryembodiment, various changes and modifications may be suggested to oneskilled in the art. It is intended that the present disclosure encompasssuch changes and modifications as fall within the scope of the appendedclaims.

None of the description in this application should be read as implyingthat any particular element, step, or function is an essential elementthat must be included in the claim scope. The scope of patented subjectmatter is defined only by the claims. Moreover, none of the claims isintended to invoke 35 U.S.C. § 112(f) unless the exact words “means for”are followed by a participle.

What is claimed is:
 1. A system for locating an occupant within avehicle, the system comprising: a communication interface configured to:receive motion data generated by a seatbelt motion sensor operablyconnected to a seatbelt for a seat in the vehicle, and receive motiondata of a wearable device of the occupant; and at least one processorconfigured to: compare the seatbelt motion data with the wearable devicemotion data from a same period of time, and identify that the occupantis located at the seat in the vehicle based on a result of thecomparison.
 2. The system of claim 1, wherein the at least one processoris configured to identify and configure personalization options for theoccupant at the seat based on a user profile of the occupant in responseto identification that the occupant is located at the seat.
 3. Thesystem of claim 1, wherein the at least one processor is configured toassociate the occupant with the vehicle based on location information ofthe vehicle and location information of the wearable device of theoccupant.
 4. The system of claim 3, wherein; the communication interfaceis configured to receive, from the vehicle, a notification that the seatin the vehicle is occupied; and the at least one processor is configuredto identify wearable device motion data for any occupant associated withthe vehicle in response to receipt of the notification.
 5. The system ofclaim 1, wherein the at least one processor is configured to identify apair of correlated wearable device and seatbelt motion patterns fromamong a plurality of wearable device motion patterns and seatbelt motionpatterns for a plurality of occupants of the vehicle.
 6. The system ofclaim 1, wherein: the seatbelt motion data and the wearable devicemotion data comprise acceleration data; and the at least one processoris configured to compare an acceleration pattern of the seatbelt with anacceleration pattern of the wearable device.
 7. The system of claim 1,wherein the system is located in one of the vehicle and a server for acloud computing system.
 8. A vehicle for locating an occupant, thevehicle comprising: one or more seats; one or more seatbelts for the oneor more seats, respectively; one or more seatbelt motion sensorsoperably connected to the one or more seatbelts, respectively; acommunication interface configured to: receive motion data generated byone of the seatbelt motion sensors for one of the seatbelts, send theseatbelt motion data to a server for comparison with motion data of awearable device of the occupant of the vehicle from a same period oftime, and receive information indicating that the occupant is located ata seat of the one seatbelt; and at least one processor configured toidentify personalization options for the occupant at the seat based on auser profile of the occupant in response to identification that theoccupant is located at the seat.
 9. The vehicle of claim 8, wherein thecommunication interface is configured to send, to the server, locationinformation for associating occupant with the vehicle based on location.10. The vehicle of claim 8, further comprising: one or more seatsensors, wherein the at least one processor is configured to identifythat the seat is occupied based on an output of the one or more seatsensors, and the communication interface is configured to send, to theserver, a notification that the seat in the vehicle is occupied.
 11. Thevehicle of claim 8, wherein the one seatbelt comprises: a buckleincluding the one seatbelt motion sensor configured to sense a motionpattern of the one seatbelt to generate the seatbelt motion data, and atransceiver configured to transmit the seatbelt motion data to thecommunication interface.
 12. The vehicle of claim 11, wherein the one ormore motion sensors comprise at least one of an accelerometer, agyroscope, and a magnetic field sensor.
 13. The vehicle of claim 8,further comprising: at least one display and at least one speakerassociated with the seat, wherein the processor configured to providethe personalization options for the occupant at the seat via the atleast one display and the at least one speaker.
 14. A method forlocating an occupant within a vehicle, the method comprising: receivingmotion data generated by a seatbelt motion sensor operably connected toa seatbelt for a seat in the vehicle; receiving motion data of awearable device of the occupant; comparing the seatbelt motion data withthe wearable device motion data from a same period of time; andidentifying that the occupant is located at the seat in the vehiclebased on a result of the comparison.
 15. The method of claim 14, furthercomprising identifying personalization options for the occupant at theseat based on a user profile of the occupant in response to identifyingthat the occupant is located at the seat.
 16. The method of claim 14,further comprising associating the occupant with the vehicle based onlocation information of the vehicle and location information of thewearable device of the occupant.
 17. The method of claim 16, furthercomprising: receiving, from the vehicle, a notification that the seat inthe vehicle is occupied, wherein receiving the motion data of thewearable device of the occupant comprises identifying wearable devicemotion data for any occupant associated with the vehicle in response toreceipt of the notification.
 18. The method of claim 14, whereinidentifying that the occupant is located at the seat in the vehiclecomprises identifying a pair of correlated wearable device and seatbeltmotion patterns from among a plurality of wearable device motionpatterns and seatbelt motion patterns for a plurality of occupants ofthe vehicle.
 19. The method of claim 14, wherein: the seatbelt motiondata and the wearable device motion data comprise acceleration data; andthe comparison includes comparing an acceleration pattern of theseatbelt with an acceleration pattern of the wearable device.
 20. Themethod of claim 14, wherein the method is performed by at least one ofthe vehicle and a server for a cloud computing system.